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Abstract Inexact Newton regularization methods have been proposed by 
Hanke and Rieder for solving nonlinear ill-posed inverse problems. Every such 
a method consists of two components: an outer Newton iteration and an inner 
scheme providing increments by regularizing local linearized equations. The 
method is terminated by a discrepancy principle. In this paper we consider 
the inexact Newton regularization methods with the inner scheme defined by 
Landweber iteration, the implicit iteration, the asymptotic regularization and 
Tikhonov regularization. Under certain conditions we obtain the order optimal 
convergence rate result which improves the suboptimal one of Rieder. We in 
fact obtain a more general order optimality result by considering these inexact 
Newton methods in Hilbert scales. 
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1 Introduction 

Inverse problems arise whenever one searches for unknown causes based on 
observation of their effects. Driven by the requirements from huge amount of 
practical applications, the field of inverse problems has undergone a tremen- 
dous growth. Such problems are usually ill-posed in the sense that their solu- 
tions do not depend continuously on the data. In practical applications, one 
never has exact data, instead only noisy data are available due to errors in the 
measurements. Even if the deviation is very small, algorithms developed for 
well-posed problems may fail, since noise could be amplified by an arbitrarily 
large factor. Therefore, the development of stable methods for solving inverse 
problems is a central topic. 
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In this paper we consider the stable resolution of nonlinear inverse problems 
which mathematically can be formulated as the nonlinear equations 

F{x)^y, (1.1) 

where F : ^{F) C A" M> 3^ is a nonlinear Frechet differentiable operator 
between two Hilbert spaces X and y whose norms and inner products are 
denoted as || • || and (•, •) respectively. We use F'{x) to denote the Frechet 
derivative of at x £ T^{F) and use F'{x)* to denote the adjoint of F'{x). We 
assume that (jl.ip has a solution in the domain 'D{F) of F, i.e. F(a;^) = y. 
Let be the only available noisy data of y satisfying 

\\y'~y\\<s (1.2) 

with a given small noise level S > 0. Due to the intrinsic ill-posedness, regular- 
ization methods should be employed to produce from a stable approximate 
solution of (jl.ip . 

Many regularization methods have been considered in the last two decades. 
Due to their straightforward implementation and fast convergence property, 
Newton type regularization methods are attractive for solving nonlinear in- 
verse problems. In [8] we considered a general class of Newton type methods 
of the form 

Xn+i - 2;„ + 5t„ iF'ixn)*F'ix„)) F'ix„y - F{x„)) , (1.3) 

where xq is an initial guess of x'^ , {tn} is a sequence of positive numbers, and 
{gt} is a family of spectral filter functions. The scheme (|1.3p can be derived 
by applying the linear regularization method defined by {gt} to the linearized 
equation 

F'{xn)ix-xn) = y^ -F{xn) 

which follows from (II. ip by replacing y by y^ and F{x) by its linearization 
F(xn) + F'{xn){x — Xn) at Xn- When the sequence {tn} is given a priori 
with suitable property, we showed in that, under the discrepancy principle, 
the methods are convergent and order optimal. We also considered in [S] the 
methods in Hilbert scales and obtained the order optimal convergence rates. 

In the definition of the Newton type methods (II. 3p . one may determine 
the sequence {tn} adaptively during computation. Motivated by the inexact 
Newton methods in [I] for well-posed problems, Hanke proposed in [4 his reg- 
ularizing Levenberg-Marquardt scheme for solving nonlinear inverse problems 
with {tn} chosen to satisfy 

\\y^ - F{Xn) - F'{Xn){Xn+l " X„)|| = T^\\y^ - F{Xn)\\ 

at each step for some preassigned number ry e (0,1) and with the discrepancy 
principle used to terminate the iteration. Rieder generalized the idea in [4] 
and proposed in [12] (see also [10]) a general class of inexact Newton methods; 
every such a method consists of two components: an outer Newton iteration 
and an inner scheme providing increment by regularizing local linearized equa- 
tions. When the inner scheme is defined by an iterative method, the number 
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of iterations is determined adaptively which has the advantage to avoid the 
over-solving of the hnearized equation that may occur when the inner scheme 
is terminated a priori. The convergence rates of inexact Newton regulariza- 
tion methods were considered in |13) but only suboptimal ones were derived. 
It is a longstanding question whether the inexact Newton methods are order 
optimal. Important progress has been made recently in [5] where the regu- 
larizing Levenberg-Marquardt scheme is shown to be order optimal. In this 
paper we consider the inexact Newton regularization methods in which the 
inner schemes are defined by applying various linear regularization methods, 
including Landweber iteration, the implicit iteration, the asymptotic regular- 
ization and Tikhonov regularization, to the local linearized equations and show 
that these methods are indeed order optimal by exploiting ideas developed in 
[51I^[TU]. We even consider these methods in Hilbert scales and derive the or- 
der optimal convergence rates. Our theoretical results confirm those numerical 
illustrations in [1^113) . 

This paper is organized as follows. In Section 2 we formulate the methods 
precisely and state the main results on the order optimal convergence rates. 
In Section 3 we show that these methods are well-defined, and prove that the 
error decays monotonically. In Section 4 we complete the proof of the the main 
result by deriving the order optimal convergence rates. 

2 Main results 

The inexact Newton regularization methods are a family of methods for solv- 
ing nonlinear ill-posed inverse problems. Every such a method consists of two 
components, an outer Newton iteration and an inner scheme providing incre- 
ments by regularizing local linearized equations. An approximate solution is 
output by a discrepancy principle. 

To be more precise, the method starts with an initial guess xq € 'D{F). 
Assume that Xn is a current iterate, one may apply any regularization scheme 
to the linearized equation 

F'(x„)u = - F{xn) (2.1) 

to produce a family of regularized approximations {u„(t)}. One may choose 
tn to be the smallest number i„ > such that 

\\y^ - F{xn) - F'(a;„)u„(t„)|| < r^Wy^ - F(a;„)|| (2.2) 

for some preassigned value Q < rj < 1. The next iterate is then updated 
as Xn+i = a^n + Un{tn)- The outer Newton iteration is terminated by the 
discrepancy principle 

\\y^ -F{xn,)\\<r6 <\\y^ ~F{xn)\\, Q<n<ns (2.3) 

for some given number r > 1. This outputs an integer ng and hence Xn^ which 
is used to approximate the exact solution x^. 
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The convergence rates of the inexact Newton regularization methods have 
been considered in P^I13| . It has been shown that if 

for some < /i < 1/2, then there is a number < fiQ < fi such that 

||X„, -X^W^ O(j2(M-M0)/(l+2p)) 

which is only suboptimal. It is a long-standing question whether the inex- 
act Newton regularization methods are order optimal. Important progress has 
been made recently in [S] where the regularizing Levenberg-Marquardt scheme 
is proved to be order optimal. 

In this paper we will consider the inexact Newton regularization methods 
in which the inner schemes are defined by applying Landweber iteration, the 
implicit iteration, the asymptotic regularization, or Tikhonov regularization 
to the linearized equation (12. ip and show that these methods are indeed order 
optimal. For these four methods, u„(t) are defined by 

Unit) = gt {F'{xn)*F'{xn)) F'{x„y {y^ - F{x„)) 

with the spectral filter functions {gt} given by 

[t] /I \-i 

5*(A)= ^(1-A)^ 5^(1 + A)-^ j{l-e-''), - + a] (2.4) 

j=0 3 = 1 ^ ^ 

respectively, where [i] denotes the largest integer not greater than t. 

We need the following standard condition which is known as the Newton- 
Mysovskii condition (see [2,). 

Assumption 2.1 (a) There exists Kq > such that 

\\[F'{x) ~ F'{z)]h\\ <Ka\\x-z\\\\F'{z)h\\, Vh e X 

for all x,z £ Bp{x'^) C 'D{F), where Bp{x'') denotes the ball of radius p > 
with center at . 

(b) F is properly scaled so that ||i^'(x)|| < 6* < 1 for all x G -Bp(a;^). 

The order optimality of these four inexact Newton regularization methods 
is contained in the following result. 

Theorem 2.1 Let F satisfy Assumvtion \2.1l let t > 2 and < rj < 1 be 

such that Trj > 2, and let xq £ Bp{x^). If Ko\\xo — x^\ is sufficiently small, 
then the inexact Newton regularization methods with the inner scheme defined 
by Landweber iteration, the implicit iteration, the asymptotic regularization, 
or Tikhonov regularization are well-defined and terminate after ns = 0(1 -f 
log(5|) iterations. If in addition, xq — x'' = {F' {x'')* F'{x^ ))'-'' u for some u G 
Af{F' {x'^))^ C X and < /i < 1/2 and if Kq\\ll!\\ is sufficiently small, then 
there holds ^ 

\\xns - a^^ll < C||a;||TW(5T+fe 

for some constant C independent of 6 and \\uj\\. 
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We will not give the proof of Theorem 12.11 directly. Instead, we will prove 
a more general result by considering these four inexact Newton regulariza- 
tion methods in Hilbert scales. Let L be a densely defined self-adjoint strictly 
positive linear operator in X satisfying 

\\x\\^ <l{Lx,x), xeV{L) 

for some constant 7 > 0, where denotes the domain of L. For each t £ M, 
we define Xt to be the completion of r\'^^^'D{L'^) with respect to the Hilbert 
space norm 

\\xh:= Wx\\. 

This family of Hilbert spaces {Xt]te^ is called the Hilbert scales generated by 
L. The following are fundamental properties (see 3 J: 

(a) For any — cx)<(7<r<oo, AVis densely and continuously embedded 
into Xq with 

<7''"'l|a;||r, x€Xr, (2.5) 

(b) For any —00 <p<q<r<oo there holds the interpolation inequality 

< ||x|||^||x|ir^, x(^Xr. (2.6) 

{c) li T : X y \s a, bounded linear operator satisfying 

m\\h\\^a<\\Th\\<M\\h\\^a, hex 

for some constants M > m > and a > 0, then for the operator A := TL^" : 
X ^ y with s> —a there holds for any |^| < 1 that 

c(^)l|/^ll-.(a+.) < \\{A*Ar'^h\\ < c(;.)||/i||_.(,+,) (2.7) 

on V{{A*Ay/'^), where A* := L-'T* : 3^ ^ A" is the adjoint of A and 

c{i>) := mm{m'^ , M'^} and c{h') — ma:ii{m'^ , M'^}. 

We will consider the inexact Newton regularization methods in which the 
inner schemes are defined by applying Landweber iteration, the implicit it- 
eration, the asymptotic regularization, or Tikhonov regularization in Hilbert 
scales to the linearized equation (|2.ip . Now we have 

Unit) = gt {L-^'F'ixnYF'ixn)) L-^'F'{xnY (/ - F(x„)) (2.8) 

with gt defined by (|2.4p . where s e M is a suitable chosen number. The iter- 
ative solutions are defined by x„+i = Xn + Un(tn) with t„ > chosen to be 
the smallest number satisfying (12.21) . The iteration is then terminated by the 
discrepancy principle (12.31) to output an approximate solution x^ ■ 

We will use Xns , constructed from these four inexact Newton regularization 
methods in Hilbert scales, to approximate the true solution x^ of (jl.ip and 
derive the order optimal convergence rate when xq — x^ € X^ with s < fi < 
b + 2s. We need the following condition on the nonlinear operator F. 
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Assumption 2.2 (a) There exist constants a > and < m < M < oo such 
that 

m\\h\\^a < \\F'ix)h\\ < M\\h\\^a, hex 
for all X G Bp{x^). 

(b) F is properly scaled so that \\F'{x)L^^\\x^y < < 1 for all x G 
Bp{x''), where s > —a. 

(c) There exist 0</3<l,0<fe<a and Kq > such that 

\\F'{x)-F'iz)\U_,^y<Ko\\x~zf 

for all X, z G Bp{x'^). 

This condition was first used in [11] for the convergence analysis of the 
nonhnear Landweber iteration in Hilbert scales. It was then used recently 
in [7] and [5] for nonlinear Tikhonov regularization and some Newton-type 
regularization methods in Hilbert scales respectively. One can consult [TTJ[7] 
for several examples satisfying Assumption 12.21 

Theorem 2.2 Let F satisfy Assunivtion \2.S\ with s > {a — b)/(3, let t > 2 and 
< rj < 1 be such that rrj > 2, and let xq G T^iF) be such that j''\\xo—x''\\s < p. 
If Kq\\xo — a;^||f is sufficiently small, then the inexact Newton regularization 
methods with the inner scheme defined by Landweber iteration, the implicit 
iteration, the asymptotic regularization, or Tikhonov regularization in Hilbert 
scales are well-defined and terminate after ns = 0{1 + \ \ogS\) iterations. If, 
in addition, xq — x^ € A'^ for some s < /i < 6 + 2s and Kq\\xo — x^\\^ is 
sufficiently small, then there holds 

\\Xn,-X^\\r < C\\xo~X^\\t^S^ 

for all r G [— a, s], where C is a constant independent of S and \\xo — x^\p. 

The proof of Theorem l2 . 21 will be given in the next two sections. Here some 
remarks are in order. 

Remark 2.1 When the inner scheme is defined by the asymptotic regulariza- 
tion or Tikhonov regularization, there is flexibility to choose i„ to satisfy 

r^iy - Fixn)\\ <\\y^ - Fixn) - F'{xnMn)\\ < mWv^ - Fixn)\\ 

with some numbers < 771 < 772 < 1- Furthermore, we only need t > 2 and 
rrji > 1 in the convergence analysis. 

Remark 2.2 When s > {a — b)/ P, the same order optimal convergence rate in 
Theorem 12.21 holds for xq — x^ G with s < p, < b + 2s which can be seen 
from the proof of Lemma in Section 4. 

Remark 2.3 If the Frechet derivative F'{x) satisfies the Lipschitz condition 

\\F' {x) ~ F' {z)\\ < Ko\\x ~ z\l x,zeBp{x^), 

then Assumption 12.21 (c) holds with 6 = and /3 — 1, and thus, for these 
inexact Newton regularization methods in Hilbert scales with s > a, the order 
optimal convergence rates hold for a;o — a;^ G Xf^ with s < fj, < 2s. 
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Remark 2.4 We indicate how Theorem 12. II can be derived from Thcorem l2.2l 
First, we note that Assumption 12. II fa) imphes 

\\F{x) ~ F{z) - F\z)[x -z)\\< \k^\\x z\\\\F\z)[x - z)\\ 

for all e Bpix^). One can then follow the proofs in Section 3 to show 
that, if G Bp{x^) and -ftro||2;o — is sufRciently small, then these inexact 
Newton regularization methods are well-defined and 

\\Xn+i - X^W < \\Xn ~ X^\, n = 0, • • • , ^5 - 1 

which implies x„ £ Bp{x^) for Q < n < ns. By shrinking the ball Bp{x'^) if 
necessary, we can derive from Assumption l2 . II fa) that there exist two constants 
< Co < Ci < cx) such that 

C4F'{z)h\\<\\F'{x)h\\<Ci\\F'{z)h\\, hex (2.9) 

for all x,z e Bp{x'^). This implies that all the operators F'{x) have the same 
null space J\f as long as x G Bp{x^). By the condition of Theorem l2.1l we have 
xq—x^ G A/"^. By the definition of {xn} we also have a;„+i—a;„ eTZ{F'{xn)*) C 
Af'^ for n = 0, ■ • ■ ,715 — 1. By considering the operator G{z) := F{z + xq) if 
necessary, we may assume = 0. Therefore x\xn G for n = 0, • • • ,ns, 
and we may consider the equation (jl.ip on . Consequently we may assume 
N = {0}, i.e. each F'{x) is injective for x S Bp{x'^). 

Now we introduce the operator L := (F'(a;^)*_F'(x^))^^/^ which is clearly 
densely defined self-adjoint strictly positive linear operator in X satisfying 

\\xf<0{Lx,x), xeV{L). 

From (113 it follows that Co||/i||-i < \\F'{x)h\\ < Ci||/i||_i which implies 
Assumption 12.21 (a,) with a = 1. Moreover, from Assumption 12.11 (b) it follows 

for x,z e i3p(a::^ )that 

\\[F'{x)~-F'iz)]\\;,_,^y ^ \\[F'ix)-F'{z)]L\\;,^y < Ko\\x ^ z\\\\F' {z)L\U^y . 

Since implies \\F'{z)L\\x^y < Ci, Assumption O (c) holds with 6=1 
and /3 = 1. Since TZ{{F'{x'')F'{x^)y) = A's^, Theorem [H] follows immediately 
from Theorem 12.21 with s = 0. 

3 Monotonicity of the error 

We start with a simple consequence of Assumption 12.21 which will be used 
frequently. 

Lemma 3.1 Let F satisfy Assumvtion \2. 2\ and let x, : z e Bp{x^). Ift > then 
\\Fix)-F{z)-F'{z)ix-z)\\ < -±^Ko\\x~z\\f^\\x-z\\7f^. (3.1) 

//, in addition, t > (a — h)/ (3, then 
\\F{x) - F{z) - F'{z){x -z)\\< ^y/^+^-i^ollx - z||f lla; - (3.2) 
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Proof From Assumption 12.21 (c) and the identity 

F{x) - F{z) - F'{z)(x - z)^ I [F'{z + t{x - z)) - [x - z)dt 

Jo 

it follows immediately that 

\\Fix) - F{z) - F'{z){x -z)\\< Y^K,\\x ^ zf\\x ~ z\U. (3.3) 
With the help of the interpolation inequality (12.61) we have 

||x-z||<||.x-^||,^||x-2||!t' and ||2;-z||_,<||a:-z||f^||x-z||!J. 

This together with p.3p gives p.ip . If, in addition, t > {a — b)/f3, then we 
have [t{l + l3) + b]/{a + t) > 1. Thus, by using ||x-z||_a < 7"+* ||a; - z||t which 
follows from the embedding (12. 5L we can derive (j3.2p immediately from (|3.ip . 

□ 

In this section we will use the ideas from [HI51ITU] to show that the four 
inexact Newton regularization methods in Hilbert scales stated in Theorem 
12.21 are well-defined and for the error term 

Cn ■ — Xn X^ 

there holds ||e„+i||s < ||en||s for n = 0, • • • ,ns ~ 1. We will use the notation 

T:=F'{x^), Tn-.^F'ixn), A:^TL-' and A„ := r„L-^ 
It follows easily from the definition (12. 8p of {u„(t)} that 

Unit) = L-^gtiA:An)A*^ {y' ~ Fix,,)) (3.4) 

and 

- F{x„) - TnUnit) = niAnAl) {y^ - F{xr,)) , (3.5) 

where rt{X) :— 1 — Xgt{X) denotes the residual function associated with gt. For 
the spectral filter functions given in (j2.4p . it is easy to see that limt^oo Tt{X) = 
for each A > 0. This implies that 

hm \\y' -F{xn)-TnUn{t)\\ = \\PniA^)^{y' - F(xn))\\, (3.6) 

where Ph(^a„)^ denotes the orthogonal projection of y onto 7?,(A„)^, the or- 
thogonal complement of the range TZ{An) of An- 

Lemma 3.2 Let F satsify Assumvtion \2.2\ with s > (o — b)/ /3, let t > 1 and 
< r] < 1 satisfy tt] > 1, and let xq S T^iF) be such that 7''||eo||s < P- Assume 
that Ko\\eo\\^ is sufficiently small. If \\y^ — F(a:„)|| > tS and ||e„||s < ||eo||s, 
then tn is well-defined and tn > cq for some constant cq > independent of n 
and S. 
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Proof From (j2.5l) and the given conditions it follows that ||e„|| < 7*||e„||s < 
7*||eo||s < P which implies Xn & -Bp(a;^). Since ||e„||s < ||eo||s < oo implies 
L^Cn € X, we have 

WPiziA^viy' - FixnM < \\y' -F{xn)+AnL'en\\ = - ^^(x„) + T„e„||. 

In order to show that t„ is well-defined, in view of p.6p it suffices to show 

||2/*-^^(a:„)+T„e„|| <77||/-F(a:„)||. (3.7) 

Since s > (a — b)//3, we can use (|1.2p and p.2p in Lemma [XT] to derive 

\\y' ~ F{x,,)+T„eJ < 5 + -±^Y^+b-Ko\\e,£\\e,,\\^, 

Now by using Assumption 12.21 fa). ||e„||s < ||eo||s and tS < \\y^ — F{xn)\\, we 
obtain with C = Y'^^''''' /[{I + l3)m] that 

11/ - F{x^) + T„e„|l < -\\y' - F(x„)|| + Ci^o||eo||f ||T„e„|| 

T 

< (^i + CXo||eo||f) \\y'~F{x^)\\ 

+ CKo\\eofs\\y^ - F{x„) + T„e„\\. 

Since tt] > 1, we therefore obtain (j3.7p if -fColleolU sufficiently small. 

For the inner scheme defined by Landweber iteration or the implicit itera- 
tion in Hilbert scales, it is obvious that i„ is an integer with t„ > 1. For the 
inner scheme defined by the asymptotic regularization or Tikhonov regular- 
ization in Hilbert scales, we have 

ry||/-F(a;„)|| = \\y' - F{xr,) ~ T^^uMW = ||rt„(A„4*)(y^ - F(a;„))|| 

where ^((A) — e^*^ or rt{X) = (1 + tX)~^. Since ||A„|| < 1, we can obtain 
either e^*" < 77 or (1 + i„)^^ < 77. Therefore i„ > log(l/?7) or t„ > 1/r/ — 1. □ 

Lemma 3.3 Let F satisfy Assumvtion \2. 2\ with s > {a ~ b)/l3, let t > 2 and 
< 77 < 1 be such that rrj > 2, and let xq £ T^{F) be such that 7''||eo||s < p. 
// ifolleolif is sufficiently small, then the four inexact Newton regularization 
methods in Hilbert scales stated in Theorem \2.2\ are well-defined and terminate 
after ns < 00 iterations, and 

ns — l 

^ i„||/-F(a;„)||2<C2||eo||2 (3.8) 

n=0 

for some constant C2 > 0. Moreover 

\\Xn+l ~ X^'Ws < \\Xn - X^Ws (3.9) 

for 0, - ■■ ,ns - 1. 
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Proof We will prove this result for the four inexact Newton methods case by 
case. 

(a) We first consider the inexact Newton method with inner scheme defined 
by Landweber iteration in Hilber scales. We first show the monotonicity (|3.9p . 
We may assume ns > 1. Let < n < ns and assume that ||e„||s < ||eo|U. By 
the definition of ns we have \\y^ — F{xn)\\ > t6. It follows from Lemma [3.21 
that tn is a well-defined positive integer. Let Un.k '■= Un{k) for each integer k. 
Then u„^o — and 

Un,k = Un,k-l + L^'^'^T* {y^ - F{Xn) - T„U„,fc_i) 

for fc = 1. ■ • • ,i„. Recall that Xn+i = a;„ + Un.t„ - Therefore, in order to show 
||e„+i||s < ||e„||s, it suffices to show 

||e„ + u„,/c||s < ||e„ + M„,fe-i||s, A: = l,---,i„. (3.10) 

We set Zn^k =y^ ~ F{xn) - TnU^^k- Then u^^k - Un,k-i = L"^T^z„,fc_i and 
thus 

||e„ + u„, fells - ||e„ + u„,fe-i||s 

= 2(e„ + U,i,fe-1, U,i,fe - Un,k-l)s + \\Un,k " Un,k-l\\l 
= {Un.k — Un^k-~l,Un,k + Uruk-1 + 2e„)s 
= {Zn.k-l,TniUn,k + Un,fe-1 + 2e„)) . 

According to the definition of z„_fe one can see 

Tn{un,k + Un,k-i + 2e„) = -z„,fe - Zn,k-i + 2{y^ - F{x„) + r„e„). 

Therefore 

||e„-l-u„, fells - ||e„ + u„,fe-i||s 

= -(^«,fe-l,2«,fe) - ||z„^fe-l|P + 2{zn^k-i,y^ - F{xn) + T„e„). 

Observing that ^^B) and rt(A) = (1 - A)[*l imply z„,fe = (/ - AnA;y{y^ - 
F{xn)), we have (z„,fe-i, 2:„,fe) > 0. Hence 

||e„+u„, fells - ||e„ + u„,fe_i||s 

< -||z„,fe-i|| (||z„^fe_i|| -2||2/''-F(2;„)+T„e„||). 

Since tt] > 2, we can pick < 770 < 77/2 with riyo > 1. By using Assumption 
12.21 tS < \\y^ — F{xn)\\ and ||e„||s < ||eo||s, we can derive as in the proof of 
Lemma [3.21 that if XolleoHf is sufficiently small then 

IK/ - Fix^) + T„e„|| < 7?o||/ - F{xn)\\. 

On the other hand, by the definition of i„ we have ||2;„_fe_i|| > ri\\y^ — F(a;„)||. 
Therefore 

||e„ + Un,k\\l - ||e„ + u^,k-i\\l < -£o\\y^ - Fi^nW, (3.11) 
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where Sq :— 'q['q — 2rio) > 0. This in particular imphes p.lOp and hence 
||en+i||s < l|6n||s- An induction argument then shows the monotonicity re- 
suh (1531) . 

Moreover, it foUows from p. lip that 

t„ 

||e„+i||s - ||e„||^ = (||e„ + u„,fe||^ - ||e„ + w„,fe_i ||^) 

k=l 

<-eotn\\y' -F{x,,)\\''. 

Consequently 

which shows (|3.8p . Since i„ > 1 and \\y^ — F(a;„)|| > t5 for < n < ns, one 
can see that ng must be finite. 

(b) For the inexact Newton method with inner scheme defined by the 
implicit iteration in Hilbert scales, all t„ must be positive integer and with the 
notation Un^k '■= Un{k) we have u„_o — and 

Un,k = + {L^" + T*Tny^T* (jj^ - F{x„) - T„U„^fe_i) . 

Let Zn.k - F{xn) ~ T^Un.k- We have from (^3)1 and rt{\) = (1 + A)"!*! 

that Zn^k ^ {I + AnA*^y^Zn,k-i and Un,k - Un,k-i = L~'^''T*Zn,k- Thus 

||en + Un, fells - Ikn + Wn,fe-l|ls 

— {'^n^k ^n,fe— 1 ; ^n,fe ^" ^n,fe— 1 ^" 2e^)s 
= {Zn,k,Tn{Un,k + Mn,fe-1 + 2e„)) 

= {zn,k, -Zn,k - Zn,fe-i + 2(?;* - F(x„) + r„e„)). 
Note that (z„,fe, > ||zn,fe|P- We then obtain 

||e„ + u„,fe||s- ||e„ + u„,fc„i||2 < -2||z„,fe|| (||z„,fc|| - Hy'' - F(x„) + T„e„||) . 
By using ||v4„|| < 1 and the definition of t„, we have 

lk„,fe|| > ^lk«,fe-i|| > iry||/-F(a:„)||, fc = 1, • • ■ ,i„. 
Since r?] > 2, we can obtain 

||e„ + u„, fells - ||e„ + Un,fe-i||s < -i^'nil - 2%)||y'' - F{xnW- 

for k = 1, • • • ,tn when XoHeoHs is sufficiently small, where < 770 < ?7/2 is 
such that Trj{) > 1. This together with an induction argument implies p.8p 
and (1231). 
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(c) For the inexact Newton method with inner scheme defined by the 
asymptotic regularization in Hilbert scales, is the solution of the ini- 

tial value problem 

jur,{t) = L-^'T* (y^ - F(x„) - T„M„(t)) , t > 0, 

Un{0) = 0. 

Therefore, with z„(i) :— — F{xn) — TnUn{t) we have 

^||e„ + u„(<)||^ = 2 (^^""(^)''^" + Un{t)^ = 2 (z„(t),r„(e„ +u„(i))) 
= 2(z„(t), -z„(t) + / - F{xn) + r„e„) 



< 



-2||z„(i)|| (||z„(i)|! - ||/-F(x„) + r„e„|l) 



According to the definition of t„ we have ||z„(t„)|| ~ ri\\y^ — F{xn)\\ and 
||z„(t)|| > ri\\y^ — F{xn)\\ for < t < Since T77 > 1, we therefore obtain 

^||en + un{t)\\l < -277(77 - 77o)||y^ - F{XnW, 0<t<tn 
at 

if ifolleollf is sufficiently small, where < 770 < 7; is such that r77o > 1. In view 
of Un{0) = and Xn+i = Xn + Un{tn), wc obtain 

||e„+i||? - ||e„||2 < -277(77 - m)tu\\y' - F{x^)\\\ 

This implies (I3.8P and p.9p immediately. 

(d) For the inexact Newton method with inner scheme defined by Tikhonov 
regularization, we have 

u^it) = (t-iL^^ + r„*r„)"'T,:(/ - F{x^)). 

We first observe that 

- Iknlls < 2||x„+i - XnWl + 2(.x„+i - a;„,e„)s 
— 2(x^i_i_i Xji^ Xji-\-i Xji -\- 67^)5. 

Let z„ = — F{xn) — Tn{xn+i—Xn)- We have from (|3.5p andrt(A) = (l+iA)~^ 
that Un{tn) = tnL^'^^T*Zn and hence a;„+i — Xn ~ tnL^'^^T^Zn- Therefore 

||e„+i|jj - ||e„||^ < 2tn (z„,r„(a;„+i - x„ + e„)) 

= 2i„ (z„, -z„ + (77'' - -F(a:;„) + T„e„)) 

< -2t„||z„|| (||z„|| - 1127* - F{xn) + T„e„||) . 

By the definition of i„ we have ||z„|| = ri\\y^ — F(a;„)||. Since r77 > 1, we can 
obtain 

\\en+l\\l - WenWl < -MV - V0)tn\\y' ~ FiXn)f 

if Xolleollf is sufhciently small, where < 770 < 77 is such that r77o > 1. This 
implies (^5]) and ([23. □ 
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Remark 3. 1 The inequality (j3.8p will find its use in the proof of Lemma 14.41 
From ((XS)) . t„ > Co > 0, and the fact \\y^ - F{xn)\\ > rJ for < n < ns, 
it follows easily that ns = 0{5~^) which gives only a rough estimate on the 
number of outer iterations. However, we should point out that the inexact 
Newton iterations in Hilbert scales in fact terminate after = 0(1 + | log(5|) 
outer iterations. This can be confirmed by using the fact 

r]\\y^ -F{xn)\\>\\y^ -F(xn)-Tn{xn+i-Xn)\\, Q < n< (3.12) 

which follows from the definition of i„ and Xn+i — Xn + Un(tn). To see this, 
by using (|3.2p in Lemma l3. II we have 

||^'(a;„+i) - F{x„) - Tnix„+i - Xn)\\ < Ko\\Xn+l - XnW^WXn+l - a:„||_a- 

Since p.9p implies ||a;„+i— < ||e„+i||s + |[e„||s < 2 ||eo||s, from Assumption 
[22](a) we have with C 2^^^^^''^" /[{^ + I3)m] that 

||F(x„+i) - F{x„) - Tnix„+i - Xn)\\ < Ci^o || gq 1 1 f || (a;„+i - 

Therefore, if ifolleoHf is sufficiently small, then there holds ||T'„(x„+i — a;n)I| < 
2||F(a;„+i) — F{xn)\\ and consequently 

l|^^(x„+i) - F{xn) - r„(x„+i - xn)\\ < 2Cifo||eo||f ||i^(a;„+i) - ^^(a:„)||. 

(3.13) 

Combining this with p.l2p yields 

r,\\y' ~ Fixn)\\ > \\y' - F{xn+i)\\ - 2Cifo||eo||f ||F(a:„+i) - ^^(.t„)||. 

Considering 77 < 1, this in particular implies that if iiTolleolls is sufficiently 
small then 

\\y'~F{xn+i)\\ ^ 7; + 2Cj^o||eo||g ^ 1+^ ^ 
\\yS-F{xn)\\ - l-2C7^o||eo||f " 2 

Therefore for all n = 0, • • ■ , ns there holds 

ll/-n^«)ll< (^)"lly'-^^(^o)l|. 

By taking n — ns — 1 and using \\y^ — F{xn^^i)\\ > tS we obtain t6 < 
^i^ys-i iiyA- _ ^(2,^^)11 ^hich shows that ns = 0{1 + |log5|). 
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4 Proof of Theorem [2T2l 

In this section we will show the order optimality of the four inexact Newton 
method in Hilbert scales stated in Theorem 12.21 For simplicity of further ex- 
position, we will always use C to denote a generic constant independent of 

5 and n, we will also use the convention <!• < ^ to mean that ^ < for 
some generic constant C when the explicit expression of C is not important. 
Furthermore, we will use (p ^ W to mean that ^ < ^f' and <<P. 

Lemma 4.1 Under the same conditions in Lemma \3.3l there holds 

\\y' - Fix,,)\\ < ||y^-F(a;„+i)||, n = 0, • ■ • , - 1. 

Proof We first claim that there is a constant ci > such that 

cilly^ - Fixn)\\ < \\y^ - Fixn) - T^{Xn+l ~ (4-1) 

This is clear from the definition of tn when the inner scheme is defined by 
Tikhonov regularization or the asymptotic regularization. When the inner 
scheme is defined by Landweber iteration, we have rt(A) = (1 - A)[*l. Ac- 
cording to the definition of t„ and (|3.5p . we have 

r]\\y^ - F{x,,)\\ < \\y^ - F(a;„) - T„M„(i„ - 1)|| 

^\\{I-A^A*J'^-\y'~Fix^))\\. 

Since ||yl„|| < 6* < 1, we have \\{I - AnA*J~^\\ < {l-e^)-\ Therefore, using 
p.Sp again it follows 

(l-e?2),7||y^-F(x„)|| < \\iI~A,,A:r-{y'^Fixnm 

= lly'' - F{Xn) ~ Tn{Xn+l - Xn)\\ 

which shows (j4.ip with ci = (1 — 0^)r]. When the inner scheme is defined by 
the implicit iteration, we have rt(A) = {1 + A)"!*!. Thus it follows from (|331) 
and ||A„|| < 1 that 

V\\y' ~ F{xr.)\\ < ||(/ + A„A:j-*"+i(y^-F(x„))|| 
<2\\iI + A^A:)-'"iy' -Fix^m 

= 2\\y^ - F{Xn) - Tn{Xn+i - Xn)\\ 

which shows (|4.ip with ci — rj/2. 

The combination of (j4.ip and (j3.13p gives 

C^\\y'-F{x^)\\ 

<\\y'- F(x„+i)|| + Ci^olleollf ||F(x„+i) - F{x^)\\ 

< 11/ - F(x„+i)ll + CXolleollf (||y' - F[x^+,)\\ + 11/ - . 

This shows the result if iiTolleollf is sufficiently small. □ 
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For the spectral filter functions defined by (|2.4p , we have shown in [9] that 
for any sequence of positive numbers {i„} there hold 

n-1 

0<X''Y[n,{X)<{s^-s,)-', (4.2) 

k=j 
n-1 

0<X''gt,W n ^t.(A) <i,(s„-s,)-'^ (4.3) 

k=j+l 

and 

n—l n— 1 

0<A''^g,.(A) n nAX)<sl-'' (4.4) 

i=0 k=i+l 

for < < 1, < A < 1 and j — 0,1, - ■ ■ ,n — 1, where {s„} is defined by 

n-1 

So = and s„ = tj for n = 1, 2, • • • . (4-5) 

j=o 

Moreover, we have the following crucial estimate. 

Lemma 4.2 Let F satisfy Assumvtion \2.Sl let {gt} be defined by {2.4\l and 
rt{X) = l — Xgt{X), and let {tn} be a sequence of positive numbers with {sn} de- 
fined by 14.5^ . Let A = F'{x^)L^'' and for any x G Bp{x'^) let = F'{x)L^''. 
Then for - , < < 1/2 there holds 

■> Z{a-\-s) — — / 

n-1 

[A* AY n r^M*A)[gtM*A)A*~gt^{AlAMV\ 
k=j+i 

<t,is^ - s,)-"-^ Ko\\x - x^f 

/or j = 0, 1, • • • ,n - 1. 

Proof We refer to [9l Lemma 2] in which similar estimates have been derived 
for a general class of spectral filter functions. □ 

We also need the following estimate concerning the sums of suitable types 
which will occur in the convergence analysis. 

Lemma 4.3 Let {tn} be a sequence of numbers satisfying in > C2 > 0, and 
let Sn be defined by |^.5[ j. Let p > and q > be two numbers. Then 

n-1 ( 1, max{p, q} < 1, 

Y^tji^n - Sj^'sJ^^ < Cis]-P-'i I log(l + s„), max{p,g} = 1, 

J=0 [ gmax{p,g}-l^ max{p, q} > 1, 

where C3 is a constant depending only on p, q and C2 ■ 

Proof This is essentially contained in 13 Lemma 4.3] and its proof. A simplified 
proof can be found in [5J Lemma 3] . □ 
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Now we are ready to give the crucial estimates on ||e„||^ and ||Te„|| for 
< n < ns- We will exploit the ideas developed in ^EUHj. 

Lemma 4.4 Let F satisfy Assumvtion \2.2\ with s > [a — b)//3, let t > 2 and 
< T] < 1 be such that rrj > 2, let xq e T^{F) satisfy 7''||eo||s < p- If sq G X^. 
for some s < < 6+ 2s and if KQ\\eo\\^ is sufficiently small, then there exists 
a constant > such that 

||e„||^ < ClleolU and ||re„|l < ||eo|U(l + s„)~^^ 
for all n = 0, ■ ■ ■ ,ns — I. 

Proof Since s < ^ < 6 + 2s, from dST]) we have ||e„||^ - \\{A*A)^f^L''e„\\. 
Therefore, it suffices to show that there exists a constant C* > such that 

\\{A*A)^L'e„\\<C4eo\\^ and ||Te„|| < ||eo||,,(l + s„)"^ (4.6) 

for all n = 0, • ■ • ,ns — 1. We will show (|4.6p by induction. By using (j2.7l) and 
Assumption 12.21 (b) we have 

||(AM)^L^eo||<c(i-^)||eo|U 
a + s 

and 

llTeoll = ||(AM)i/2i^eo|| < ||(AM)^L^eo|| < c(^)||eo|U. 

Therefore with n = holds for C* > c(|^). Now we assume that 

is true for all < n < / for some < I < ng and want to show that it is also 

true for n — I. 

From the equation (j3.4p and Xn+i = Xn + u„(i„) it follows that 

e„+i = e„ + L-''gt,SA*^An)Al {y^ ~ F{xn)) 

= L- Vt„ (AM)L^e„ + L-'gt„ {A* A) A* {y' - F(xn) + Te^) 
+ L-^ [gaAlAn)Al - gtM*A)A*] (/ - F{x^)). 

By induction on this equation we obtain 

i-i ;-i /-I 

ei = L-^\{rt^{A*A)L^e^ + L-^Y. 11 nM* A)gtM* A)A* {y' ^ y) 

j=0 j=Ok=j+l 

/-I i-i 

+ ^"'E n n,{A*A)gtM*^)A*{y-F{x,) + Te,) 

j=0 k=j+l 

/-I ;-i 

+ E n (^*^) b*. (^;^.)^; - 9t, {A* A) A*] {y' - F{x,)) . 

j=0 k=j+l 

(4.7) 
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By multiplying (|4J)) by T := noting that A = TL^", and using the 

identity 

/-I /-I (-1 

l-A^.g,^.(A) n rtA^) = Il^t,W 

3=0 k=j+l j=0 

which follows from the relation rt(A) = 1 — \gt{X), we can obtain 



i-i 



Tei^A'[[rt^{A*A)U 



eo 



i-i 



iy' - y) 



I-\{n^{AA* 

j=o I j=0 

l-l l-l 

+ E n nMA*)9tAAA*)AA*{y-F{x,)+Te,) 

j=0 k=j + l 

i-i i-i 

+ E ^ n (^*^) [9t, {A*A,)A* - g,^ {A* A) A*] {y' - F{x,)). 

j=0 k=j + l 

(4.8) 

Since eo G A"^ with s < /i < & + 2s, by using (^7]), (021), (g^l), (g^]) and 
Lemma |4?2] we can derive from (14.71) that 



||(A*A)5T^L^ei|| 



/-I 



< calleoll,. + s (5 + ^ tj(si - Sj 



(4.9) 



where C3 = c(^^) and C is a generic constant independent of / and 5. 

Next by using again eo G with s < /i < 6 + 2s, (|2.7I) and (|4.2I) . we can 
obtain 



/-I 

Aj]rt^.(^M)L«eo 



< 



/-I 



^[]rt^(^*A)(AM) 



2(a + s) 



3=0 



{A*A)-^(^L'eo 



<C3 sup A=(»+=) lleolU 

0<A<1 \ fj^ ' 



< 



Therefore, it follows from and Lemma li?^ that 



/-I 



llTeHI < C3S, ^<"+=> lleolU + ^ + E ^^(^^ ^ " ^(^^O + ^^^"1 



b + a + 23 



+ Cj2t3isi - s,r^i^Ko\\e,\f\\y' - F(x, 
j=o 



(4.10) 
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We first use (I4.10p to derive tlie desired estimate for UTe; || . According to the 

relation ||ej||^ ^ || 2(<i+=) L^e^- 1|, we have from the induction hypotheses 
that 

l|ejt<||eot and ||re,|| < ||eolU(l + s,)" < j < Z - 1. (4.11) 

We need to estimate the terms 

||e,||, \\y'-Fixj)\\ and \\y - F{xj) + Te^l < j < Z - 1. 

For each term we will give two types of estimates, one is true for all < j < l — l 
and the other is true for < j < Z — 1. 

By using (|3.2p in Lemma [3. 1[ Assumption 12.21 fa). Lemma [3.31 and tS < 

\\y^ — F{xj)\\ for < j < rt^ we have 

\\y'~Fix,) + Te,\\ < 5 + \\y - F{x,) + Te^W < S + CKo\\ej\fje,\U 
<^\\y'-F{x,)\\+CKo\\eofjTe,\\. 
This shows for < j < ng that 

\\y' - Fix,)\\ < ^ (1 + CKoWeof,) \\Te,l (4.12) 
\\y' - F{x,)\\ > (1 - Cifolleollf) ||Te,||. (4.13) 

The inequalities (I4.12p . (|4.13p and Lemma WTH imply that if i^o||eo||f is suffi- 
ciently small then 

re,l|<||Te,+i||, 0<j<ns-l. (4.14) 
Consequently, we have from (j4.12p and (|4.14p that 

l|y'-^^(a;,)|| < ||re,+i||, 0<j<ns^l. (4.15) 
This together with (j4.1ip gives 

\\y' -F{xJ)\\<\\eo\\^.s;^, < j < Z - 1. (4.16) 

Next we estimate \\y ~ F{xj) + Tej\\. We have from (|3.2I) in Lemma [3.11 
Assumption (a), and (|4.1ip that 

||y - F(x,) - Te.-ll < Xo||e,||^||e,||_, < Xo||eo||^||re,||. 

Therefore, it follows from (j4.14p that 

\\y-F{x,)~Te,\\<KMi\\Te,+^\, 0<j<l-\. (4.17) 

On the other hand, by using p.ip in Lemma [5TT] and Assumption 12.21 fa), we 
have 

a(l + g)-b ^(l + ll) + h 

\\y-F{x,) + Te,\\<K4e,\\, \\e,\U:^^ 

<i^o||e,|U \\Te,\\-^^ 
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Therefore, it follows from (I4.14p and (|4.1ip that 

)j(l + /3) + b 

\\y-F{x,)-Te,\\<KMl+0s':^^^^^'^ , 0<j</-l. (4.18) 

For the term ||ej||, we first have from the interpolation inequality (|2.6p . 
Lemma 13. 3[ and Assumption 12.21 (a) that 

l|e,ll<||e,|U^||e,||*<||eol|.^l|Te,l|*. 
With the help of (|4.13p we then obtain 



e, 



.■||<l|eo||ril2/'-i^(x,)||^, 0<j<Z-l. (4.19) 



On the other hand, by using the interpolation inequality (j2.6p and Assumption 
12.21 (a) we also obtain for < j < Z — 1 that 

l|e,ll<||e,||^||e,||^ <||e,||^||re,||^. 
This together with (|4.14D and (14. lip gives 



l|e,ll <||eolUs^;r^>, Q<j<l~l. (4.20) 

Now we use (liT^ . (|iT7)) and (HT^ with j = ^ - 1 and use (HJS)) 
and (I4.20p for < j < ^ - 1, we then obtain from (|4.10l) that 

__a±jJ_ n(l + g) + b 

+ Ci^o||eo||^||Te,|| + CKoWeoWf" tff^ \\y' - F(xz_i)||* ||TezI| 

Since fj, > s > {a ~ b)/P, we can use Lemma 14.31 to derive that 

llre.ll < (C3 + CKoWeolf,) HeoH.^r"^ + -5 + CKo\\eofjTci\\ 
+ CKo\\eo\\P'tff^\\y' -Fixi^,)\\^\\Tei\\. 

Recah that dSS]) in Lemma lO implies ti-i\\y^ - F{xi-i)\\'^ < ||eo||^ Since 
s > {a — h)/ (3 and > Co > 0, we have 

t^\\y'-F{xi^,)\\^^ < (i,_i||/-F(xz_i)f)^t,^ < lleolU*. 
Therefore, noting ||eo||s < ||eolUj we obtain 

llTeHI < (C3 + CXolleoll^) lleolU^r"^ + ^ + CXo|ieo||^||Te,||. (4.21) 
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Since I < ng, we have from the definition of ns and (14.121) that 

S < i\\y^ _ F{xO\\ < 7^ (1 + C^olleoll^) \\Tei\\. (4.22) 
Combining this with (|4.21l) gives 

WTeiW < (c3 + CKoWeof,) \\eo\\,s;^ + (^-^ + CKo\\eo\f^ \\Tei\\. 
Recah that r > 2. Therefore, if i^'olleoH^ is sufficiently smah, then we have 

\\Tei\\ < lleolUs; 

Since Z > 1 and s; > > cq, we have 1 + s/ < (1 + l/co)s;. Therefore 

||Te,|l < C*||eo||^(l + s/)~^S^ if we choose C, > 2c3(l + l/co)(r - 1)/(t - 2). 

Finally we will use (j4.9p to show the desired estimate for L'^ei\\. 
Since we have verified the estimates for ||re;||, the estimates (j4.16p . (j4.18l) and 
(|420)) therefore can be improved to include j = I ~ 1; this is clear from the 
above argument. Consequently we have from (|4.9p that 

\\{A*A)^L'ei\\ 

It then follows from Lemma [4.31 that 

\\{A*A)^^L'ei\\ < {c2+CKo\\eof^) \\eo\\^ + sf^ S. 
With the help of (|4.22p and the estimate on ||rei||, we obtain 

||(A*A)TOL^eH| < (C3 + Ci^olleoli;!) ||eolU + + CK„\\eo\\1)\\eo\\,. 

Since t > 2, we thus obtain \\{A* A)^^^ L'' ei\\ < C,||eo||p for any C, > 
4c3(t — 1)/(t — 2) if iiTolleoll^ is sufficiently small. The proof is therefore com- 
plete. □ 

Now we are ready to complete the proof of Theorem 12.21 the main result 
in this paper. 



Proof of Theorem \2.SX Considering Lemma 13.31 and Remark 13.11 it remains 
only to derive the order optimal convergence rates. When ng = 0, the proof 
is standard. So we may assume ns > 0. From Lemma 14.41 it follows that 
||era5-ilU ^ IkolU- By using Lemma 14.11 and the definition of ng we have 
\\y^ — F(a:„^_i)|| < S, which together with (j4.13p implies that ||e„^_il|_a < 
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||Te„^_i|| < 6. Therefore, from the mterpolation mequality (I2.6P it foUows 
that 

In view of (I3.9p in Lemma 15751 we consequently obtain ||e„5 ||s ^ lleoH^^'' ■ 
By using the definition of ns and (ll.2p we have ||y — -F(a;„j)|| < (1 + t)(5. 
Observing that (j3.2p in Lemma 13.11 and (I3.9P in Lemma 13.31 imply 

||Te„J| < ||z/-F(x„J|| + ||y-i^(x„J + re„Jl 

< \\y-Fixr.,)\\+CKo\KAfjTe„A\ 

< ||t/-F(x„J||+CXo||eo||f||re„,||. 

Thus, if ivTolleolls < ifo||eolU is sufficiently small, then ||re„J| < \\y-F{xns)\\- 
Consequently ||e„^||_a < ||re„^|| < S. Now we can use again the interpolation 
inequality (12. 6p to derive for all r G [—a, s] that 




The proof is therefore complete. □ 



5 Conclusions 

Inexact Newton regularization methods have been suggested by Hanke and 
Rieder in [3] and [H] , respectively, for solving nonlinear ill-posed inverse prob- 
lems. The convergence rates of these methods have been considered in [1^113) , 
the results however turned out to be inferior to the so-called order optimal 
rates. For a long time it has been an open problem whether these inexact 
Newton methods are order optimal, although the numerical illustrations in 
[T^ITB] present strong indication. 

Important progress has been made recently in [5] where the regularizing 
Levenberg-Marquardt scheme is shown to be order optimal affirmatively. In 
this paper we considered a general class of inexact Newton methods in which 
the inner schemes are defined by Landweber iteration, the implicit iteration, 
the asymptotic regularization and Tikhonov regularization. By establishing 
the monotonicity of iteration errors and deriving a series of subtle estimates, 
we succeeded in proving the order optimality of these methods. We also ex- 
tended these order optimality results to a more general situation where the 
inner schemes are defined by linear regularization methods in Hilbert scales. 
Our theoretical findings confirm the numerical results in [T^fTB] . 
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